CityMii - An integration and interoperable middleware to manage a Smart City

Modern cities are supported by multiple heterogeneous IT systems deployed and managed by distinct agents. In general, those systems use old, dependent and non-standardized technologies, which make them legacy and incompatible systems. As smart cities are moving toward a fully centralized management approach, the lack of integration among systems raises several problems. Since they are independent, it is not easy to correlate information from different systems and put it together to work in order to achieve application goals. The collaboration among different systems enables an agent to offer new functionalities (services or just information about the city) that cannot be provided by any of these systems working as individual entities. The goal of this paper is to propose an integration middleware to support the management of Smart Cities in a dynamic, transparent and scalable way. The proposed middleware intends to support interoperability among different systems operating in a city.info:eu-repo/semantics/publishedVersio

Machine Learning techniques for energy consumption forecasting in Smart Cities scenarios

Over the past few years, the number of sensors spread across cities has significantly increased. This led to an exponential growth in data volume, which can only be treated with Big Data techniques. Having such a large amount of generated data, turns possible to apply machine learning techniques more accurately, with the goal of making data predictions over time, finding anomalies, performing classification, among other tasks. This article aims to show the application of machine learning techniques, using a variant of the Recurrent Neural Networks, the Long Short-Term Memory (LSTM), in order to predict city\u27s energy consumptions for the near future. This forecast will support municipal entities decisions, helping them to improve the managing of energy consumptions and budgets

Comparative Analysis of Process Mining Tools

In the current context of availability of large amounts of data (Big Data), its underlying value can be, frequently, devalued. However, there are several tools that allow to extract knowledge from data. Among other information, this knowledge can lead to improve processes or detect any failure during their execution. This work intends to compare several process mining (PM) tools, using different techniques. For each tool, the best scenario in the discovery of processes is found and the respective results are evaluated. The results showed that Disco is the simplest and most intuitive tool to use. Along with ProM, it also allows a complete analysis, without the need for theoretical knowledge concerning PM or programming. PM4Py, on the other hand, is a free framework that allows great customizations for all functionalities. So it is ideal for professionals with knowledge in PM needing more adjusted implementation or integration with other applications. From a cost perspective, either PM4Py or ProM are free. The use of PM4Py can be complemented by ProM for compliance verification

Selecção de planos de mineração de dados de utilização da web

Tese de Doutoramento em Informática - Especialidade de Inteligência ArtificialA descoberta de conhecimento em dados de clickstream, relativos à interacção de indivíduos com sítios Web, está a assumir um papel, cada vez mais, preponderante, englobando uma audiência crescente de agentes de decisão ao longo da organização. A intenção subjacente reside em auxiliar as organizações a atingir as metas estabelecidas para os sítios que promovem e a maximizar as oportunidades emergentes da Web, explorando dados recolhidos, por inerência e de forma implícita, que, apesar de serem complexos e vastíssimos, constituem uma fonte extremamente rica e abrangente acerca do comportamento dos visitantes. No entanto, o desenvolvimento e a aplicação desses processos de mineração de dados são actividades que se revestem de grande complexidade, especialmente para utilizadores sem experiência e conhecimentos profundos neste domínio. Uma forma de combater este desafio consiste em proporcionar ferramentas consentâneas, capazes de assistirem os utilizadores na condução desses processos, procurando, deste modo, contribuir para a simplificação e acréscimo dos níveis de eficácia e de produtividade destas iniciativas. A estratégia defendida, para este efeito, desenrola-se em torno da gestão e reutilização, ao nível da organização, do conhecimento adquirido a partir da experiência prática, referente à resolução de problemas concretos que facultaram, no passado, processos bem sucedidos de mineração de dados de clickstream. O âmbito organizacional de tal estratégia visa, principalmente, fomentar um uso sinergético de recursos da organização, integrando os contributos de vários colaboradores e colocando as potencialidades deste tipo de mineração ao alcance e ao serviço de todos os seus membros, inclusive dos utilizadores mais inexperientes. O trabalho apresentado nesta dissertação descreve um sistema fundamentado no paradigma de raciocínio baseado em casos, o qual foi concebido com o propósito de assistir os utilizadores em duas formas primordiais: (i) captura, organização e armazenamento, num repositório de casos partilhado, do conhecimento acerca de exemplos úteis e bem sucedidos de processos de mineração de dados de clickstream; (ii) selecção dos planos de mineração alternativos e mais adequados, para solucionar um problema específico de análise de dados neste âmbito, dada uma descrição de alto nível desse mesmo problema. O sistema proposto foi implementado através de uma aplicação Web protótipo, a ser explorada ao nível da organização, consolidando o conhecimento respeitante a exemplos de exercícios de mineração de utilização da Web, numa base de casos centralizada. O sistema integra e retira benefícios de recursos relacionados da organização, suportando uma abordagem semi-automática de aquisição de conhecimento, a partir dos seguintes tipos de origens: fontes de dados da organização; documentos normalizados em formato PMML, produzidos por ferramentas de extracção de conhecimento e representativos de actividades de mineração concretizadas; informação complementar, obtida por meio de interacção com o utilizador. No apoio à resolução de problemas, o sistema actua a partir de um conjunto de requisitos da análise e de características dos dados de clickstream disponíveis, e, com base no conhecimento relativo à aplicação de métodos de mineração e de outras operações, sugere planos de mineração alternativos e apropriados para os dados em causa e para o fim a que a análise se destina. Tais planos são apresentados ao utilizador através de descrições gerais, acompanhadas por informação suplementar e por referências para detalhes explicativos da sua implementação pragmática.Discovering knowledge from clickstream data, related to the interaction of individuals with Web sites, is playing an increasingly important role, reaching a growing number of decision makers across the organization. The intention behind this is helping organizations to achieve the goals of the promoted sites and to maximize the latent opportunities of the Web, exploring data inherently and implicitly collected, which are huge and complex, yet a very rich and comprehensive source of visitants’ behavior insights. However, developing and applying such mining processes are very complex tasks, especially to users without deep knowledge and experience in this domain. One way to tackle this challenge is by building tools, capable of assisting users within such processes realization, in order to simplify these initiatives and to increase theirs efficacy and productivity levels. The defended strategy regarding such assistance relies on managing and reusing, at corporative level, the knowledge acquired from the practical experience in solving concrete problems, which had provided successful clickstream data mining processes in the past. This corporative-wide perspective mostly aims at favoring an synergetic use of the organization resources, bringing up together the contributions of distinct collaborators and making available the potentialities of this kind of mining to all members, including the inexperienced users. The work presented in this dissertation describes a system founded on the case based reasoning paradigm. This system was devised with the purpose of assisting users in two main ways: capturing, organizing and storing, on a shared case repository, the knowledge about successful and useful clickstream data mining processes; selecting the most suited and alternative mining plans, to solve a specific clickstream data analysis problem, given an high level description of such problem. The proposed system was implemented as a prototype Web-based application, to be explored at corporate level, consolidating the knowledge about Web usage mining processes examples on a centralized case base. The system integrates and takes advantage from related corporative resources, supporting a knowledge acquisition semi-automated approach from the following types of origins: corporative data sources; standard documents in PMML format, supplied by knowledge extraction tools and representing the mining activities accomplished; complementary information, obtained through user interaction. When advising problem solving, the system acts, taking the characteristics of the available clickstream data and the analysis requirements, and based on the acquired knowledge about applying data mining and other operations, suggests the most appropriate alternative mining plans to the data and the analysis at hands. The plans are deployed as overviews, complemented by additional information and by links to practical implementation details

Analyzed and processed Waste data within the scope of Smart Cities

Within the domain of Smart Cities, specifically within the waste theme, the goal is to create Common Information Models, resorting to Big Data, IoT and Machine Learning technologies, so that they are integrated into a data management platform, allowing a city council, which will be one of the customers of this platform, to make decisions according to the data provided, all with the aim of optimizing the waste management processes adjacent to the city. The datasets are inherently linked to the waste from the city of Austin, and these data are “real” in nature, meaning that they were and are recorded by the collection company that operates in the city

Analysis and Real-time Data of Meteorologic Impact on Home Solar Energy Harvesting

Solar energy production increased in the world from 0 TWh in 1965 to 724.09 TWh in 2019. Solar energy is adopted as a source for residential renewable energy sources because, besides Biomass sources, it’s the only one that can be installed and maintained at home. Operating efficiency is an important consideration when evaluating the application of photovoltaic panels (PV) technology. A real-time system monitoring is required to analyse the current production and understand the impact of the weather conditions on PV production. This paper extends the literature on the residence solar energy harvesting subject, by providing a scalable architecture that can be used as starting point on data analysis on PV panels efficiency and how weather conditions impact energy production. A dataset was collected related to PV panel energy production, the residence energy consumption and that’s reading weather conditions. Wind intensity and direction, temperature, precipitation, humidity, atmospheric pressure and radiation were weather conditions analysed. Moreover, this data was analysed and interpreted in order to evaluate the pros and cons of the architecture as well as how the weather impacted the energy production

Comparison of Semi-structured Data on MSSQL and PostgreSQL

The present study intends to compare the performance of two Data Base Management Systems, specifically Microsoft SQL Server and PostgreSQL, focusing on data insertion, queries execution, and indexation. To simulate how Microsoft SQL Server performs with key-value oriented datasets we use a converted TPC-H lineitem table. The data set is explored in two different ways, firsts using the key-value-like format and second in JSON format. The same dataset is applied to PostgreSQL DBMS to analyse performance and compare both database engines. After testing the load process on both databases, performance metrics (execution times) are obtained and compared. Experimental results show that, in general, inserts are approximately twice times faster in Microsoft SQL Server because they are injected as plain text without any type of verification, while in PostgreSQL, loaded data includes a validating process, which delays the loading process. Moreover, we did additional indexation tests, from which we concluded that in general, data loading performance degrades. Regarding query performance in PostgreSQL, we conclude that with indexation, queries become three or four percent faster, and six times faster in Microsoft SQL Server.info:eu-repo/semantics/publishedVersio

An Advertising Real-Time Intelligent and Scalable Framework for Profiling Customers Emotions

The advertising industry is continuously looking up for effective ways to communicate to customers to impact their purchasing. Usually, profiling them is a time-consuming offline activity. Therefore, it becomes necessary to reduce costs and time to address consumers’ needs. This work proposes a scalable framework enabled by a Machine Learning (ML) model to profile customers to identify their emotions to help to drive campaigns. A multi-platform mobile application continuously profiles consumers crossing the front stores. Profiling customers according to their age and hair color, the color of their eyes, and emotions (e.g. happiness, sadness, disgust, fear) will help companies to make the most suitable advertisement (e.g. to predict whether the advertising tones on the front store are the adequate ones). All that data are made available in web portal dashboards, wherein subscribers can take their analysis. Such results from the analysis data help them to identify tendencies regarding the culture and age, and drive companies to fit front stores accordingly (e.g. to discover the right tones for the season). This framework can help to develop new innovative cost-effective business models at scale by driving in real-time the advertisements to a huge number of consumers to maximize their impact and centralizing the data collected from a large number of stores to design future campaigns.info:eu-repo/semantics/publishedVersio

Analysis and real-time data of meteorologic impact on home solar energy harvesting

Solar energy production increased in the world from 0 TWh in 1965 to 724.09 TWh in 2019. Solar energy is adopted as a source for residential renewable energy sources because, besides Biomass sources, it’s the only one that can be installed and maintained at home. Operating efficiency is an important consideration when evaluating the application of photovoltaic panels (PV) technology. A real-time system monitoring is required to analyse the current production and understand the impact of the weather conditions on PV production. This paper extends the literature on the residence solar energy harvesting subject, by providing a scalable architecture that can be used as starting point on data analysis on PV panels efficiency and how weather conditions impact energy production. A dataset was collected related to PV panel energy production, the residence energy consumption and that’s reading weather conditions. Wind intensity and direction, temperature, precipitation, humidity, atmospheric pressure and radiation were weather conditions analysed. Moreover, this data was analysed and interpreted in order to evaluate the pros and cons of the architecture as well as how the weather impacted the energy production.info:eu-repo/semantics/publishedVersio

Process Mining – case study in a process of IT incident management

This paper describes a process mining techniques application to event data from a particular process of Huf Portuguesa, a company that produces automotive components. Huf uses state-of-the-art technology and maintains a close connection with research, seeking to keep itself updated, through the acquisition of new knowledge, technologies and procedures. The process analyzed registers incidents and actions taken to resolve them. There is a planned procedure and, through process mining tools, namely ProM, Disco and Celonis, the processes were analyzed to see if they are in compliance or if there are deviations. The interactions between resources and departments were also analyzed. The results obtained made it possible to understand the behavior of the processes in question, the relationship between departments and the interaction between people, in this context. These results can be used so that failures can be minimized and productivity and quality can be maximized